Machine beats experts: Automatic discovery of skill models for data-driven online course refinement
نویسندگان
چکیده
How can we automatically determine which skills must be mastered for the successful completion of an online course? Large-scale online courses (e.g., MOOCs) often contain a broad range of contents frequently intended to be a semester’s worth of materials; this breadth often makes it difficult to articulate an accurate set of skills and knowledge (i.e., a skill model, or the QMatrix). We have developed an innovative method to discover skill models from the data of online courses. Our method assumes that online courses have a pre-defined skill map for which skills are associated with formative assessment items embedded throughout the online course. Our method carefully exploits correlations between various parts of student performance, as well as in the text of assessment items, to build a superior statistical model that even outperforms human experts. To evaluate our method, we compare our method with existing methods (LFA) and human engineered skill models on three Open Learning Initiative (OLI) courses at Carnegie Mellon University. The results show that (1) our method outperforms human-engineered skill models, (2) skill models discovered by our method are interpretable, and (3) our method is remarkably faster than existing methods. These results suggest that our method provides a significant contribution to the evidence-based, iterative refinement of online courses with a promising scalability.
منابع مشابه
Machine Beats Experts: Automatic Discovery of Skill Models for Data-Driven Online Courseware Refinement
How can we automatically determine which skills must be mastered for the successful completion of an online course? Large-scale online courses (e.g., MOOCs) often contain a broad range of contents frequently intended to be a semester’s worth of materials; this breadth often makes it difficult to articulate an accurate set of skills and knowledge (i.e., a skill model, or the QMatrix). We have de...
متن کاملMachine learning 2.0 Engineering data driven AI products
ML 2.0: In this paper, we propose a paradigm shift from the current practice of creating machine learning models that requires months-long discovery, exploration and “feasibility report” generation, followed by re-engineering for deployment, in favor of a rapid 8 week long process of development, understanding, validation and deployment that can executed by developers or subject matter experts ...
متن کاملDrug Discovery Acceleration Using Digital Microfluidic Biochip Architecture and Computer-aided-design Flow
A Digital Microfluidic Biochip (DMFB) offers a promising platform for medical diagnostics, DNA sequencing, Polymerase Chain Reaction (PCR), and drug discovery and development. Conventional Drug discovery procedures require timely and costly manned experiments with a high degree of human errors with no guarantee of success. On the other hand, DMFB can be a great solution for miniaturization, int...
متن کاملTopicResponse: A Marriage of Topic Modelling and Rasch Modelling for Automatic Measurement in MOOCs
This paper explores the suitability of using automatically discovered topics from MOOC discussion forums for modelling students’ academic abilities. The Rasch model from psychometrics is a popular generative probabilistic model that relates latent student skill, latent item difficulty, and observed student-item responses within a principled, unified framework. According to scholarly educational...
متن کاملRelevance vector machine and multivariate adaptive regression spline for modelling ultimate capacity of pile foundation
This study examines the capability of the Relevance Vector Machine (RVM) and Multivariate Adaptive Regression Spline (MARS) for prediction of ultimate capacity of driven piles and drilled shafts. RVM is a sparse method for training generalized linear models, while MARS technique is basically an adaptive piece-wise regression approach. In this paper, pile capacity prediction models are developed...
متن کامل